Thumb access area for one-handed touchscreen use

ABSTRACT

Embodiments of the invention provide a system for reconfiguring touch controls on a device, such as a cell phone, in reaction to a user&#39;s changing mode of operation. For example, a standard display of icons that works well in two-handed operation may not work as well for one-handed operation where the user tries to operate the controls with a thumb of the holding hand. In such a case, a device can detect that the user is trying to operate the controls with a thumb of the holding hand and the controls can be reconfigured to be more suitable for thumb operation. Reconfiguration can also occur in other modes of operation, such as when the user is using a stylus, multiple fingers of a second (non-holding hand), fingers on both hands, etc. In some embodiments, detection of the mode of operation can be automatic such as where a camera takes an image to determine the mode.

CROSS REFERENCES TO RELATED APPLICATIONS

This application is a continuation of the following application, U.S.patent application Ser. No. 13/492,478, entitled HANDHELD DEVICE WITHRECONFIGURING TOUCH CONTROLS, filed on Sep. 8, 2014 with is acontinuation of U.S. Pat. No. 8,863,042, entitled HANDHELD DEVICE WITHTOUCH CONTROLS THAT RECONFIGURE IN RESPONSE TO THE WAY A USER OPERATESTHE DEVICE, issued on Oct. 14, 2014, which claims priority from U.S.Provisional Patent Application Ser. No. 61/590,284; entitled USINTERFACE USING DEVICE AWARENESS, filed on Jan. 24, 2012, which ishereby incorporated by reference as if set forth in full in thisdocument for all purposes.

BACKGROUND

Embodiments of the invention relate generally to handheld electronicdevices and more specifically to a touch interface for such a devicethat reconfigures in response to the way a user operates the device.

Portable handheld computing devices such as cell phones, personaldigital assistants (PDAs), audio players, cameras, global positioningsatellite (GPS) devices, games, etc., are more commonly using a touchinterface. In a touch interface, the user is presented with images ofcontrols such as buttons, icons, links, sliders, bars, boxes, etc. Theuser is able to manipulate the controls by touching the display screenon which the controls are displayed. Typically a user holds the devicein one hand and uses a finger on the other hand to tap or slide acontrol on the screen. However, often a user may try to operate thedevice in a different manner, such as one-handed operation where thethumb of the holding hand is used to operate the controls. In suchone-handed operation it may be difficult or impossible to operate thecontrols easily.

SUMMARY

Embodiments of the invention provide a system for changing the location,number, type or other operation of touch controls. For example, astandard display of controls that works well in two-handed operation maynot work well for one-handed operation where the user tries to operatethe controls with a thumb of the holding hand. In such a case, a devicecan detect that the user is trying to operate the controls with a thumbof the holding hand and the controls are reconfigured to be moresuitable for thumb operation. Reconfiguration can also occur in othermodes of operation, such as when the user is using a stylus, multiplefingers of a second (non-holding hand), fingers on both hands, etc. Insome embodiments, detection of the mode of operation can be automaticsuch as where a camera takes an image to determine the mode. Or the usercan change modes manually with a control or by voice, touch, gesture,device movement or other commands.

In one embodiment a method for reconfiguring controls shown on a touchscreen display is disclosed. The touch screen display is included in adevice, wherein the reconfiguring allows one or more of the controls tobe more accessible for operation of a thumb of a hand holding thedevice. The method includes displaying at least one control in a firstposition on the touch screen display; accepting a signal to indicatethat reconfiguring for a thumb mode of operation is desired; andchanging the display in response to the signal so that the at least onecontrol is moved to a thumb access area, wherein the at least onecontrol is more accessible for operation by the thumb than when the atleast one control was in the first position.

Another embodiment provides a method for reconfiguring controls shown ona touch screen display, wherein the touch screen display is included ina device, wherein the reconfiguring allows one or more of the controlsto be more adapted for use with a stylus, the method comprising:displaying at least one control in a first position on the touch screendisplay; accepting a signal to indicate that reconfiguring for a stylusmode of operation is desired; and changing the display in response tothe signal so that the at least one control is adapted for the stylusmode.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a first illustration of a thumb mode of operation;

FIG. 2 is a second illustration of a thumb mode of operation;

FIG. 3 is a third illustration of a thumb mode of operation;

FIG. 4 is a fourth illustration of a thumb mode of operation;

FIG. 5 is a fifth illustration of a thumb mode of operation;

FIG. 6 is a sixth illustration of a thumb mode of operation;

FIG. 7 is a seventh illustration of a thumb mode of operation;

FIG. 8 shows a thumb access area for an exemplary thumb on an exemplarydevice;

FIG. 9 shows a particular embodiment where icons and system controls ofFIG. 1 have been relocated to an access area; and

FIG. 10 illustrates basic hardware that can be suitable for use withembodiments disclosed herein

DETAILED DESCRIPTION OF EMBODIMENTS

FIGS. 1-7 are first discussed to establish a basic understanding ofoperating a device's touch screen with a thumb of a holding hand.

FIG. 1 shows a typical cell phone held in the left hand. Cell phone 100includes a touch screen that occupies substantially all of the frontface of the device. Various controls are displayed on the touch screensuch as icon 116 which can be pressed to launch an applicationcorresponding to the icon. In FIG. 1, 20 icons are displayed in a gridof 5 rows by 4 columns. The navigation controls above the icon grid at114 include the words “APPS” and “WIDGETS.” These words are actuallybutton controls along with the icon in the same row to the right. Thetopmost controls above the navigation controls at 112 show thewell-known icons of wireless connectivity, cell signal strength, batterycharge level and time. The topmost control bar can be swiped down toreveal more control options. Finally, system buttons at 118 include“BACK,” “HOME” and “TASK SWITCHER.” Other control features may bepresent such as allowing the user to swipe left or right across the icongrid to move to a new page, or new set, of icons.

Although specific controls are shown and discussed herein, it should beapparent that any number, type and arrangement of controls can be usedin different situations. The specific details of the provided Figuresare intended to provide examples to illustrate features of variousembodiments of the invention. Features which may be used in any othersuitable devices and interfaces.

It should be apparent that by using the index finger, or other finger,of the user's other hand—in this case the right hand (not shown)—itwould be a simple matter to press, swipe or otherwise activate ormanipulate any of the controls on the touch screen of FIG. 1. However,FIG. 2 shows an example of the user using thumb 200 of the holdinghand—in this case the left hand—to activate controls on the touchscreen. Although a particular hand may be discussed it should beapparent that much of the discussion can apply to the symmetricalsituation in which the hands are switched. In FIG. 2, the thumb isattempting to tap on icon1-2, that is, the icon in row 1, column 2. Thisicon is at the top row and second from the left. Note that whileaccessing icon1-2 is possible, it is generally not as easy as using theindex finger of right hand. Also, the part of the thumb that iscontacting the touch screen is not the tip of the thumb but actually theleft side of the thumb which can cause the user's press to be lessaccurate or precise.

FIG. 3 shows the thumb extending to reach icon1-4. Although icon1-4 isjust reachable in FIG. 3, this may not always be the case depending onthe size of the device, layout of icons, size of the user's hand andthumb, etc. Also, the vertical position of the phone with respect to theuser's holding hand (i.e., along the direction A-A′) plays an importantrole in the ability of the thumb to access different parts of the touchscreen effectively. Various embodiments of the invention that take thesefactors into consideration are discussed in more detail below.

FIG. 4 shows the thumb pressing icon4-3. This is a fairly easy accessfor the device and hand size and position shown in FIG. 4. Note that inthis position the thumb is able to press with more of the tip than theside in contrast to the case of FIG. 2.

In FIG. 5, the thumb attempts to press icon4-1. This is a difficultmovement and even if the thumb can make the press, the part of the thumbthat contacts the touch screen is a rather large area that extends alongthe entire side of the thumb from the first knuckle to the tip. Thereare ways to improve the thumb's range and reduce the contact area suchas by loosening the grip and sliding the palm of the hand away from thephone as shown in FIG. 6, however, this makes for an unstable grip andis still an awkward pressing movement with the thumb.

Finally, FIG. 7 shows the thumb pressing the HOME system control button.In order to do this, the grip had to be changed to slide the hand downin the direction A-A′. The little finger is placed underneath the bottomof the phone for stability. Such re-positioning movements are oftennecessary when using the thumb of the holding hand to operate the devicebut can be awkward, uncomfortable and time-consuming.

FIG. 8 shows thumb access area 200 for an exemplary thumb on anexemplary device. Although access area 200 is defined with visibleboundaries, it should be apparent that these are not hard boundaries butmerely loosely define the workable or efficient area in which the thumbcan act to more easily press, swipe or otherwise operate a control. Thesize, shape and definition of the optimal access area can vary dependingon factors such as user hand size; skeletal and muscular shape, strengthand other properties; size of the device, type of controls, userpreferences, etc. In general, for thumb operation, the access area has aretracted curve 206 which is the curve traced along a line easilycontacted by the thumb by moving the thumb from top 202 to bottom 204while the thumb is retracted. Extended curve 208 is a similar curvetraced while the thumb is extended. Top 202 and bottom 204 are definedas the limits of top and bottom swing of the thumb while still allowingthe thumb to make an effective, controlled and accurate press or contactwith the touch screen. In general, a thumb access area is an area on atouch screen in which thumb pressing by the holding hand is improvedover areas outside of the access area.

In general, in a standard or two-handed mode of operation the entiretouch screen area may be used for controls similar to what is shown inFIGS. 1-7. In a one-handed mode of operation, one or more of thecontrols may be moved to or toward the access area in order to allowbetter operation of the control by a thumb of the user.

FIG. 9 shows a particular embodiment where the icons and system controlsof FIG. 1 have been relocated to access area 200. Icon 106, for example,is in the upper right of the access area that corresponds to theupper-right of the icon grid of FIG. 1. Note that other controls in thestandard configuration are not moved such as the controls at 112 and114, discussed previously.

The access area can be automatically defined by the device such as byhaving a default access area. The device can also request that the usertrace an access area and use the resulting trace. The device can alsouse the front-facing camera 130 to image the movements of the thumb andgenerate the access area by observation. In one embodiment, theinterface can be automatically switched from a standard mode (e.g.,default or two-handed mode) of control arrangement to the one-handedmode (i.e., thumb mode) of control arrangement upon the device detectingone-handed operation by using the front-facing camera 130. For example,if a thumb is detected by the camera then the user interface is switchedto thumb mode. If it is determined that an index finger is approachingthe screen then the screen can be switched back to two-handed mode. Suchswitching may be delayed to take place after the first activation of acontrol so that the user does not try to press a control only to see thecontrol move away to a different location on the screen.

A user can be allowed to change the shape of the access area. Differentpatterns for the access area can be provided and the user can selectfrom among them. Updates to the access area shape can be by system orapplication updates performed by the manufacturer of the device,operating system or another entity. Access patterns can be transferredto or from other devices or users. Other ways to define, estimate ormanipulate the access area are possible.

Similar to thumb pressing, thumb swipe sensing can be modified so thatthe thumb is not required to make movements that might be too awkward.For example, given the curved, arcuate nature of the retracted curve 206and extended curve 208 (see FIG. 8) it may be beneficial to allow acurve similar to that of 206 and/or 208 rather than require a verticalline swipe as is often needed for a two-handed (non-thumb) mode.Likewise, rather than require a horizontal swipe, as in the two-handedmode, the device can allow an angled swipe with the left thumb similarto what is shown for the top and bottom thumb access area boundaries 202and 204 of FIG. 8.

A similar approach can be taken to reconfigure one or more controls onthe touch screen when it is determined that a user wants to use thedevice in a stylus mode. A stylus, such as a pen or other pen-likedevice, can allow a user to point to, touch, activate or otherwisemanipulate smaller controls, such as smaller icons, than the user wouldbe able to efficiently use with their fingertip, thumb or other digit oftheir hand. In a stylus mode, the icons can be made smaller and/orplaced closer together. The stylus mode can be entered automaticallysuch as by using the camera to capture and analyze an image to determinehow the user is trying to use the device. Or the stylus mode can beentered automatically by accepting a signal from a user control such asa button or other on-screen control, or a physical button, slider, etc.on the device. Or by using voice command, gesture or other type of userinput.

FIG. 10 shows basic hardware that can be suitable for use with functionsand embodiments disclosed herein. Although specific hardware componentsare presented, many suitable types and configuration of components arepossible. FIG. 10 merely illustrates an example of a generalized system.Subsystems or components may be added, omitted or changed from thoseshown in FIG. 10. Any suitable implementation of the functionalityattributed to the components shown can be used.

In FIG. 10, device 300 can be a cell phone or other handheld or portabledevice. Display 302 can be a liquid crystal display (LCD),light-emitting diode (LED), organic LED (OLED), active matrix OLED(AMOLED), plasma, cathode ray tube (CRT), or any other suitable type ofdisplay. Touch screen 304 is typically integrated with or adjacent tothe surface of display 302 so that when a user touches the screen thetouching is sensed and translated into a control operation or command.Touch screen 304 can be implemented by any suitable technology such asresistive, capacitive, infrared, optical imaging, surface acoustic wave,etc.

Physical controls 306 include any tactile dedicated buttons such as avolume rocker, power on/off switch, home button, etc., that may beprovided on the device. Processor 308 represents any type of processorconfiguration including electrical, optical, quantum, chemical,biological, micro-electromechanical systems (MEMS), etc. One or moreprocessors may be used to achieve the functionality describe herein.Although specific interconnections are shown in FIG. 10, any suitabletype of connections or communication links between components orsubsystems are possible.

Random Access Memory (RAM) 310 includes instructions that are executableby processor 308 to achieve the functions described herein. In general,RAM 310 can be any type of processor-readable storage device such aselectronic, electromagnetic, optical, etc. Camera 312 includes optical,infrared; still or video, or other types of cameras that may be providedin a device. As discussed above, the inclusion of a camera can make itpossible to automatically determine how the user is trying to use thedevice so that mode switching and control reconfiguration can beautomatic. Image data captured by camera 312 is transferred to processor308 for analysis. Processor 308 can determine, for example, whether auser is trying to operate on-screen controls with an index finger,thumb, stylus or other object.

Camera 312 can be used to detect gesture commands. Microphone 314 can beused to receive voice commands. Other components and/or subsystems shownas 316, 318, 320 and 322 can be used for other purposes such as tooutput audio, communicate with other devices, store information, etc.

Although embodiments of the invention have been described with respectto particular embodiments thereof, these particular embodiments aremerely illustrative, and not restrictive. Details, including cameradetection of user operations, can be found in documents incorporated byreference at the beginning of this specification.

Larger devices that may be adaptable for use with features describedherein even though the devices may be considered too large for easy“handheld” or “portable” operation. For example, tablet or slatecomputers such as the iPad™ by Apple Computer, Inc. can be used eventhough these devices are significantly larger than cell phones.

Any suitable programming language can be used to implement the routinesof particular embodiments including C, C++, Java, assembly language,etc. Different programming techniques can be employed such as proceduralor object oriented, scripts, interpreted or compiled code, etc. Theroutines can execute on a single processing device or multipleprocessors. Although the steps, operations, or computations may bepresented in a specific order, this order may be changed in differentparticular embodiments. In some particular embodiments, multiple stepsshown as sequential in this specification can be performed at the sametime.

Particular embodiments may be implemented in a computer-readable storagemedium for use by or in connection with the instruction executionsystem, apparatus, system, or device. Particular embodiments can beimplemented in the form of control logic in software or hardware or acombination of both. The control logic, when executed by one or moreprocessors, may be operable to perform that which is described inparticular embodiments.

Particular embodiments may be implemented by using a programmed generalpurpose digital computer, by using application specific integratedcircuits, programmable logic devices, field programmable gate arrays,optical, chemical, biological, quantum or nano-engineered systems,components and mechanisms may be used. In general, the functions ofparticular embodiments can be achieved by any means as is known in theart. Distributed, networked systems, components, and/or circuits can beused. Communication, or transfer, of data may be wired, wireless, or byany other means.

It will also be appreciated that one or more of the elements depicted inthe drawings/figures can also be implemented in a more separated orintegrated manner, or even removed or rendered as inoperable in certaincases, as is useful in accordance with a particular application. It isalso within the spirit and scope to implement a program or code that canbe stored in a machine-readable medium to permit a computer to performany of the methods described above.

As used in the description herein and throughout the claims that follow,“a”, “an”, and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise.

Thus, while particular embodiments have been described herein, latitudesof modification, various changes, and substitutions are intended in theforegoing disclosures, and it will be appreciated that in some instancessome features of particular embodiments will be employed without acorresponding use of other features without departing from the scope andspirit as set forth. Therefore, many modifications may be made to adapta particular situation or material to the essential scope and spirit.

The invention claimed is:
 1. A method for processing a user input from atouch screen coupled to a computing device, the method comprising:generating a display of a plurality of controls within a predefinedthumb access area on the touch screen, wherein at least one dimension ofthe thumb access area is based on a user's anatomical hand shape;accepting a signal from the touch screen to indicate a user activationof one of the plurality of controls within the thumb access area; andperforming an action associated with the activated control.
 2. Themethod of claim 1, wherein the thumb access area is an area within whicha user's thumb can act to more easily operate a control within the thumbaccess area than outside of the thumb access area.
 3. The method ofclaim 1, wherein the thumb access area is a default shape.
 4. The methodof claim 1, wherein at least one dimension of the thumb access area isbased on a user's hand size.
 5. The method of claim 1, wherein the atleast one dimension of the thumb access area includes a user's handskeletal shape.
 6. The method of claim 1, wherein at least one dimensionof the thumb access area includes a user's hand muscular shape.
 7. Themethod of claim 1, further comprising: accepting a signal from a userinput device to change the thumb area.
 8. The method of claim 1, whereinthe thumb area is defined at least in part by a retracted curve.
 9. Themethod of claim 1, wherein the thumb area is defined at least in part byan extended curve.
 10. An apparatus for processing a user input, theapparatus comprising: a touch screen for displaying images and forgenerating a signal in response to a touch from a user; a digitalprocessor coupled to the touch screen; one or more tangible mediacoupled to the digital processor and including instructions executableby the digital processor for: generating a display of a plurality ofcontrols within a predefined thumb access area on the touch screen,wherein at least one dimension of the thumb access area is based on auser's anatomical hand shape; accepting a signal from the touch screento indicate a user activation of one of the plurality of controls withinthe thumb access area; and performing an action associated with theactivated control.
 11. The apparatus of claim 10, wherein the at leastone dimension of the thumb access area includes a user's hand skeletalshape.
 12. The apparatus of claim 10, wherein at least one dimension ofthe thumb access area includes a user's hand muscular shape.
 13. Themethod of claim 10, further comprising: accepting a signal from a userinput device to change the thumb area.
 14. The method of claim 10,wherein the thumb area is defined at least in part by a retracted curve.15. The method of claim 10, wherein the thumb area is defined at leastin part by an extended curve.
 16. The apparatus of claim 10, wherein theat least one dimension of the thumb access area includes a user's handskeletal shape.
 17. One or more tangible media including instructionsexecutable by one or more digital processors for: generating a displayof a plurality of controls within a predefined thumb access area on atouch screen, wherein at least one dimension of the thumb access area isbased on a user's anatomical hand shape; accepting a signal from thetouch screen to indicate a user activation of one of the plurality ofcontrols within the thumb access area; and performing an actionassociated with the activated control.
 18. The one or more tangiblemedia of claim 17, wherein at least one dimension of the thumb accessarea includes a user's hand muscular shape.
 19. The one or more tangiblemedia of claim 17, further comprising: accepting a signal from a userinput device to change the thumb area.
 20. The one or more tangiblemedia of claim 17, wherein the thumb area is defined at least in part bya retracted curve.